Multi-Platform Intelligent System for Multimodal Human-Computer Interaction

Multimodal Human Computer Interaction

Multimodal Human-computer Interaction

While human-to-human communication takes advantage of an abundance of information and cues, human-computer interaction is limited to only a few input modalities (usually only keyboard and mouse) and provides little flexibility as to choice of communication modality. In this paper, we present an overview of a family of research projects we are undertaking at Carnegie Mellon and Karlsruhe Univers...

Describing Multimodal Human-Computer Interaction

In this paper we argue that approaches to annotate multimodal human face-to-face interaction are not suitable for current device-based human-computer interaction. Instead, existing extensions proposed to established parameters describing the interaction with spoken dialog systems are presented and discussed. A standardization activity for annotating user interactions with multimodal systems is ...

Gaze tracking for multimodal human-computer interaction

This paper discusses the problem of gaze tracking and its applications to multimodal human-computer interaction. The function of a gaze tracking system can be either passive or active. For example, a system can identify user's message target by monitoring the user's gaze, or the user could use his gaze to directly control an application or launch actions. We have developed a real-time gaze trac...

Connectionist Models in Multimodal Human-computer Interaction

We present an overvi ew of our l aboratori es' research on Mul timodal Human-Computer Interf aces. By expl oi ti ng al l avai l abl e channel s of human communi cati on we aim to i ncrease exi bi l i ty, robustness, and natural ness of human-computer i nteracti on. The i nformati on sources we process i ncl ude Speech-, Character, and Gesture Recogni ti on, Faceand Eye Tracki ng, Li preadi ng, ...